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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]An image processing device comprising: 

A magnification means to which image data is expanded. 

A filter means which filters expanded image data, and a control means to which the filtering effect of 
said filter means is changed according to magnifying power in said magnification means. 

[Claim 2]An image processing device, wherein said control means enlarges said filtering effect in the 
device according to claim 1, so that said magnifying power increases. 

[Claim 3]An image processing device when said magnifying power is below a threshold in a device 
given in either of claims 1 and 2, wherein said control means suspends operation of said filter means. 
[Claim 4]An image processing device characterized by said filter means being a median filter which 
outputs a mean value of inputted image data in the device according to any one of claims 1 to 3. 
[Claim 5]An image processing method comprising: 
An expansion step to which image data is expanded. 

A processing step which filters expanded image data according to magnifying power. 

[Claim 6]An image processing method enlarging an effect of said filtering at said processing step in a 
method according to claim 5, so that said magnifying power increases. 

[Claim 7]An image processing method not filtering at said processing step in a method given in either 
of claims 5 and 6 when said magnifying power is below a threshold. 

[Claim 8]An image processing method performing median filtering which outputs a mean value of 
inputted image data at said processing step in a method according to any one of claims 5 to 7. 



[Translation done.] 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to an image processing device and a method, especially 

filtering accompanying expansion of image data 

[0002] 

[Description of the Prior Art]Conventionally, in the digital camera etc., it has the function to expand a 

picture. Picture magnification interpolates picture element data fundamentally. 

Some problems may arise from it being what generates the pixel which is not from the first. 

[0003]The (B) case is shown in drawing_8. [ which expanded electrically the original image (A) 
drawing^^ S ° me 0rigina ' imag6S 3 Paft ° f eXpansi ° n Picture (B) are typically shown in 

[0004]In order for pixel interpolation to generate a new pixel, as shown in drawing 9 (B) a notch 
rnnn^Tu ^ 0utlme of an ex P^sion picture (jaggy). This jaggy causes image quality deterioration 
|0005JThen, convent.onally, in order to ease the jaggy of an expansion picture, processing 
magnification .mage data with a median filter is proposed. In a median filter, since the mean value of 
inputted image data .s outputted, specific data can be eliminated. The case where a median filter is 
applied to the image data shown in (B) is shown in drawing 9 (C). 
It turns out that the notched portion shown in (B) is eased. 

[0006]The processing principle of the median filter is shown in drawing 10. Two or more picture 
element data .s inputted, these are sorted according to the value, and the data which has a mean 
value is outputted. Supposing nine picture element data which carried out two-dimensional array is 
inputted by a diagram and it has a value of 15. 30. 23, 49, 108. 77, 27, 90. and 81. respectively the 
picture element data which has a pixel value of 49 which is these mean values will be filtered and 
outputted. 

[0007]The example of composition of the median filter is shown in drawing 1 1 . This median filter 
inputs and processes the seven data d0-d6. The median filter 1 0 connects two or more filter 
elements 10a in the shape of a cascade, and is constituted. Each filter element 10a is an element of 2 
input 2 output and as shown in drawingj_2. it outputs the one between two inputted data where a 
value is 'arger from an ^upside output terminal. For example, as shown intA), when the two inputted 
data A and B fills A>=B. the data A is outputted from the upper part, and the data B is outputted 
from the lower part. As shown in (B). in A<B. the data B is outputted from the upper part and the 
data A is outputted from the lower part. By combining two or more such filter elements 10a in the 
seven data d0-d6, what has a small value is divided into the lower part, and what has a large value will 
be divided into the upper part, and will be sorted according to the value after all. And a mean value 
can be outputted with outputting outside the data of the center sorted eventually. It cannot be 
overemphasized that the number of the filter elements 10a to combine is determined according to the 
data number inputted. s 
[0008] 

[Problem(s) to be Solved by the Invention]The median filter is effective in relaxation of a jaggy in 
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order to eliminate a characteristic portion, but the characteristic portion which should originally exist 
on the other hand is also eliminated, and there is also a possibility of spoiling original image data. 
[0009]the case where original image data (A) and this original image data are expanded to drawing 13 
— (B) and the case where it expands further — (C) — the case where a median filter is applied is 
shown about each. (D) is filtered to the original image data of (A), a white picture element and a black 
pixel are reversed, and original image data is spoiled. (E) is filtered to the magnification image data 
(magnifying power 8/6) of (B), and original image data is spoiled selectively in a similar manner. (F) is 
filtered to the magnification image data (magnifying power 8/6x8/6=64/36) of (C), original image data 
is maintained as it is, and it is hardly spoiled, and the jaggy is also eased. 

[0010]Thus, a median filter has a problem which spoils an original image conversely, when it applies 
uniformly [ a thing effective in jaggy relaxation ]. Especially, when magnifying power is small, image 
quality will deteriorate with a median filter that it is easy to produce the pixel pattern isolated to 
image data (the interpolation amount of a pixel will increase so much and the isolated pixel pattern 
will decrease if magnifying power is high). 

[0011]this invention is made in view of the technical problem which the above-mentioned 
conventional technology has, and comes out. There is the purpose in providing the device and method 
of also being able to control degradation of image quality and acquiring a quality expansion picture 
while easing the jaggy produced in a picture. 

[0012] 

[Means for Solving the Problem]To achieve the above objects, it is characterized by this invention 
comprising the following. 

A magnification means to which image data is expanded. 
A filter means which filters expanded image data. 

A control means to which the filtering effect of said filter means is changed according to magnifying 
power in said magnification means. 

It does not filter uniformly to an expansion picture, but image quality deterioration accompanying 
filtering can be controlled by changing the filtering effect according to magnifying power of an 
expansion picture, easing a jaggy. 

[0013]Here, it is so preferred for said control means to enlarge said filtering effect that said 
magnifying power increases. As shown in drawin g 13, when magnifying power is small, image quality 
deterioration by filtering becomes remarkable. Then, image quality deterioration accompanying filtering 
can be certainly controlled by a thing which enlarge the filtering effect, so that magnifying power 
increases and for which the filtering effect is made small, so that in other words magnifying power is 
small, enlarging the filtering effect does not necessarily mean that magnifying power and the filtering 
effect are in proportionality, so that magnifying power increases, and it means being in positive 
correlation. That is, it is contained besides in case the filtering effect increases continuously 
according to magnifying power when increasing nonsequetially or gradually. 
[0014]In one embodiment of this invention, said control means suspends operation of said filter 
means, when said magnifying power is below a threshold. Thereby, when magnifying power is as small 
as below a threshold, magnification image data will be outputted as it is, and can prevent image 
quality deterioration by filtering. Since there are also comparatively few jaggies when magnifying 
power is as small as below a threshold, image quality deterioration by a jaggy is not produced, either. 
[0015]In one embodiment of this invention, said filter means is a medianJilter which outputs a mean 
value of inputted image data. Although a mean value of inputted image data is outputted in a median 
filter, probability that the original expansion picture will be outputted becomes high by making the 
filtering effect of a median filter small. Therefore, image quality deterioration resulting from filtering 
can be controlled by making the filtering effect of a median filter small according to magnifying power, 
when magnifying power is small and there are comparatively few jaggies. Magnifying power is large, 
and since the filtering effect of a median filter also becomes large according to magnifying power 
when a jaggy appears notably, a jaggy can be eased. And since a pixel number interpolated also 
increases when magnifying power is large, image quality deterioration by a median filter is hardly 
produced, either. 

[0016]This invention provides an image processing method. In this method, it has an expansion step 
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to which image data is expanded, and a processing step which filters expanded image data according 
to magnifying power. 

[0017]Here, it is so preferred to enlarge an effect of said filtering that said magnifying power 
increases in said processing step. 

[001 8]It is preferred not to filter in said processing step, when said magnifying power is below a 
threshold. 

[0019]In said processing step, median filtering which outputs a mean value of inputted image data can 

be performed. 

[0020] 

[Embodiment of the Invention] Hereafter, the embodiment of this invention is described based on a 
drawing. 

[0021]The entire configuration figure of the image processing device concerning this embodiment is 
shown in drawing 1 . The image data obtained by CCD 12 is supplied to the digital zoom part 14. In the 
digital zoom part 14, expanding processing of the image data is carried out with a suitable magnifying 
power according to the instructions from CPU20. CPU20 can output expansion instructions according 
to a user's operation. The expanded image data is memorized by SDRAM18 by the SDRAM controller 
16. When the image data memorized by SDRAM 18 needs to be expanded further, the SDRAM 
controller 16 reads image data, supplies it to the limb 22, and expands image data electrically. Since 
there is a possibility that a jaggy etc. may arise in image data, in connection with the expanding 
processing of these series as mentioned already, the expanded image data is supplied to the median 
filter 10 next. 

[0022]Fundamentally, the median filter 10 connects the filter element 10a to a cascade like the 
conventional median filter, is constituted and outputs the mean value of two or more inputted picture 
element data. The composition of the median filter 10 is the same as the composition shown in 
drawin g 1 1 , in this embodiment, 15 picture element data is inputted and the mean value is outputted. 
[0023]Although magnification image data is supplied to the median filter 10 as it is and the mean 
value is outputted in the former, In this embodiment, CPU20 computes a total magnifying power of 
image data (it is the magnifying power which combined the magnifying power in the digital zoom part 
14, and the magnifying power in the limb 22, and). When either a digital zoom part or a limb exists, the 
function of the median filter 10 is controlled according to the magnifying power and magnifying power. 
When a total magnifying power is below a threshold about a total magnifying power as compared with 
a predetermined threshold, the median filter 10 is not more specifically operated, and inputted image 
data is outputted as it is. When a total magnifying power is below a threshold, it may be made not to 
supply magnification image data to the median filter 10. When a total magnifying power exceeds a 
threshold, the filtering effect in the median filter 10 is fluctuated according to a total magnifying 
power. That is, when magnifying power is small, according to it, the filtering effect is also made small, 
and when magnifying power is conversely large, the filtering effect is also enlarged according to it. 
The method of changing the filtering effect is mentioned further later. Expanding processing is carried 
out and the image data by which filtering processing was carried out according to a still more total 
magnifying power is changed into a JPEG format in the JPEG Cau Dick part 24, and is memorized by 
the storage 26. Or it is changed into a video signal with an NTSC encoder and LCD driver 28, and is 
displayed on LCD which is not illustrated. 

[0024]The digital zoom part 14 and the limb 22 do not necessarily need to exist separately, and are 
good also as a single function. — 

[0025]The composition of the median filter 10 in drawing 1 is shown in drawing 2. As mentioned 
above, the median filter 10 of this embodiment inputs 15 picture element data, outputs the mean 
value, and it is provided with the median filter of 15 taps. The multiplexer is provided in the preceding 
paragraph of this 15 tap median filter, the pixel signal which exists in an end among 15 pixel signals is 
suitably transposed to a specific value, and 15 tap median filter is supplied. The six picture element 
data dO which exists in an end among 15 picture element data, d4, d5, d9, d10, d14, and a total of 
seven data of the central picture element data d7 that should be processed are specifically inputted 
into a multiplexer, these are chosen suitably, and it outputs as picture element data of six ends. Any 
of the picture element data of six ends a multiplexer replaces replaces and outputs to the central 
picture element data d7 which is determined based on the switching signal from CPU20, for example, 



http://www4jpdUnpit.gojp/cgi-binA 2008/07/28 



JP,2002-044425,A [DE"^^D DESCRIPTION] 




4/5 ^— v 



should process the picture element data dO, d4, d10, and d14 based on a switching signal. The 
switching signal from CPU20 is determined according to a total magnifying power computed by 
CPU20. 

[0026] 15 picture element data which is the targets of processing with the median filter 10 is typically 
shown in drawin g 3. Two-dimensional array of the 15 picture element data is carried out, and it is set 
to dO, d1, d2, .... d14, respectively. These are picture-element-data groups located near the central 
picture element data d7 which should be processed, and the picture element data of an end is dO, d5, 
d10 (left edge part) and d4, d9, and d14 (right end section). Of course, an upper bed part, d10, d1 1, 
d12, d13, and d14 can also be made into a lower end part for dO, d1, d2, d3, and d4. 
[0027]When 15 picture element data shown in d rawin g 3 is inputted into 15 tap median filter as they 
were, the probability that the central picture element data d7 which should be processed will be 
outputted as it is is 1/15. In this case, it is as having mentioned already that there is a possibility that 
an original image may be spoiled — a white picture element and a black pixel are reversed when 
magnifying power is small. Therefore, magnifying power is comparatively small, and when it is better to 
output the central picture element data d7 which should be processed as it is, it is necessary to raise 
the priority of the central picture element data which should be processed, and to make the filtering 
effect of a median filter small. 

[0028]The case where it transposes to the central picture element data d7 which should process two 
picture element data of the picture element data d5 and d9 among the picture element data of an end 
by operation of CPU20 and a multiplexer is shown in drawing 4. In this case, the probability that the 
central picture element data which should be processed will be outputted from 15 tap median filter 
will be 3/15, The priority of the central picture element data which should be processed compared 
with drawing 3_becomes large, and the filtering effect becomes small (the probability that the central 
picture element data which should be processed will be outputted as it is becomes high). 
[0029]The case where it transposes to the central picture element data d7 which should process four 
picture element data, the picture element data dO, d4, d10, and d14, among the picture element data 
of an end by operation of CPU20 and a multiplexer is shown in drawing 5. In this case, the probability 
that the central picture element data which should be processed will be outputted from 1 5 tap 
median filter will be 5/15, the priority of the central picture element data which should be processed 
compared with drawing 3 becomes large, and the filtering effect becomes small. Even if it compares 
with the case of drawing 4, the filtering effect becomes small more. 

[0030]The case where it transposes to the central picture element data which should process six 
picture element data, the picture element data dO, d4, d5, d9, d10, and d14, among the picture 
element data of an end by operation of CPU20 and a multiplexer is shown in drawing 6. In this case, 
the probability that the central picture element data which should be processed will be outputted 
from 15 tap median filter will be 7/15, the priority of the central picture element data which should be 
processed compared with drawi ng 3 becomes large, and the filtering effect becomes small. Even if it 
compares with the case of drawin g 5, the filtering effect becomes small more. 
[0031]Thus, the degree of reduction of the filtering effect can also be enlarged by increasing the 
number which can make the filtering effect small and moreover replaces it by transposing to the 
central picture element data d7 which should process either of the picture element data of an end. 
Therefore, when easing the jaggy of an expansion picture using the median filter 10, optimal filtering 
can be performed by determining the number which CPU20 replaces suitably according to magnifying 
power, without changing hardware constitutions. — . 

[0032]The relation of a total magnifying power and the filtering effect of the median filter 10 in this 
embodiment is shown in drawin g 7. The vertical axis of a figure is the filtering effect and it is shown 
that the filtering effect (filtering degree) is so large that it goes upwards. When a total magnifying 
power is below the threshold Mth, as for the filtering effect, 0, i.e., filtering, is not performed, but 
magnification image data is outputted as it is. And when a total magnifying power exceeds the 
threshold Mth, the filtering of the median filter 10 operates, but the filtering effect is enlarged, so that 
not uniform filtering but magnifying power increases. While this eases the jaggy produced by 
expansion of a picture, the image quality deterioration by a median filter can also be prevented. 
[0033]As mentioned above, although the embodiment of this invention was described, this invention is 
not limited to this, and various change is possible for it and it can use other methods as a method of 
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changing especially the filtering effect 

[0034]For example, the filtering effect can be changed also by not transposing the picture element 
data of an end to the central picture element data d7 t but transposing to the minimum of 00 h and 
the maximum ffh of image data. 

[0035]Namely, when two picture element data is transposed to the minimum of 00 h, and the 
maximum ffh among end picture element data, respectively, When it sorts with 15 tap median filter, 
these two picture element data certainly serves as the top (a value is the largest) and the lowest (a 
value is the smallest), and is not outputted as a mean value. In other words, it becomes equivalent to 
having changed the number of picture element data except two picture element data which has the 
minimum and the maximum among 15 picture element data (tap numbers) from which it will remain, a 
mean value will be chosen and outputted out of 13 pieces, and this is the target of filtering to 13 
pieces from 15 pieces. In this case, the probability that the central picture element data which should 
be processed will be outputted as it is as a mean value will increase from 1/15 to 1/13, and the 
filtering effect can be made small. If the number of picture element data to replace is made into four 
pieces, probability will be 1/11 and can make the filtering effect small further. What is necessary is 
just to change the pixel number transposed to the minimum and the maximum according to a total 
magnifying power. 

[0036]In this embodiment, when a total magnifying power exceeds the threshold Mth, it is filtering, but 
when the threshold Mth is set to zero, namely, a picture is expanded, it can also always filter. Also in 
this case, when magnifying power is small, it cannot be overemphasized that the filtering effect is 
made small. 

[0037]In this embodiment, when a total magnifying power exceeds the threshold Mth, according to 
magnifying power, the filtering effect is fluctuated continuously, but fluctuating nonsequetially or in 
step is also possible. 

[0038]Although the image processing device of this embodiment can be applied, for example to a 

digital camera, it is not limited to this. 

[0039] 

[Effect of the Invention]According to this invention, as explained above, while easing the jaggy 
produced in an expansion picture, degradation of image quality can also be controlled and a quality 
expansion picture can be acquired. 

[Translation done.] 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[ Drawing 1 lit: is a lineblock diagram of the image processing device of an embodiment 
[Drawing 2] It is a lineblock diagram of the median filter in drawing 1. 
[Drawing 3]It is a picture-element-data explanatory view of an embodiment 

[Draw^L0_g„4]It is an explanatory view at the time of transposing two picture element data of an end to 
d7. 

[Drayying_5]It is an explanatory view at the time of transposing four picture element data of an end to 
d7. 

[Qrawing.6]It is an explanatory view at the time of transposing six picture element data of an end to 
d7. 

[Drawing_7]They are graph charts showing the magnifying power of an embodiment, and the relation 
of the filtering effect 

[ Draw ing 8]It is an expansion picture explanatory view. 

[Drawing _9]It is an explanatory view at the time of applying the jaggy and median filter of an 
expansion picture. 

[Drawing 10] It is a principle explanatory view of a median filter. 

[ Draw ing 11]It is a lineblock diagram of a median filter. 

[Drawing 12]It is an explanatory view of a filter element of operation. 

[ Draw ingJ_3]It is a picture explanatory view at the time of applying the conventional median filter. 
[Description of Notations] 

10 A median filter and 14 A digital zoom part, 20 CPU, 22 limbs. 



[Translation done.] 
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[Drawing 13] 
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